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Abstract 

In Direct Sum problems [8], one tries to show that for a given computational model, 
the complexity of computing a collection F = {fi(x\), . . . fi(xi)} of finite functions on 
independent inputs is approximately the sum of their individual complexities. In this 
paper, by contrast, we study the diversity of ways in which the joint computational 
complexity can behave when all the fi are evaluated on a common input. We fo- 
cus on the deterministic decision tree model, with depth as the complexity measure; 
in this model we prove a result to the effect that the 'obvious' constraints on joint 
computational complexity are essentially the only ones. 

The proof uses an intriguing new type of cryptographic data structure called a 
'mystery bin' which we construct using a small polynomial separation between deter- 
ministic and unambiguous query complexity shown by Savicky. We also pose a variant 
of the Direct Sum Conjecture of [8] which, if proved for a single family of functions, 
could yield an analogous result for models such as the communication model. 

1 Introduction 

A famous 'textbook' result in algorithms [10], [5] is that given a list of n integers, it is 
possible to locate both the maximal and the minimal element using — 2 comparisons; 
this is an improvement over the naive strategy of computing each separately, which would 
take In — 2 comparisons. In such a setting we say, informally, that a 'multitask efficiency' 
exists between the MAX and MIN functions, because the tasks of computing them can be 
profitably combined. We emphasize that, in contrast to 'direct sum problems', where we wish 
to understand the complexity of computing several functions (usually the same function) on 
several disjoint input variable sets, here we are interested in evaluating multiple functions 
on a common input. 

While in relatively simple examples such as MAX/MIN the multitask efficiencies that 
exist can be well-understood, for even slightly richer examples the situation becomes more 
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complex. For instance, in the 'set maxima' problem, one is given a list of n integers and a 
family of subsets Si of [n], and is asked to find the maximal element in each corresponding 
subset of the list. A significant amount of research has focused on finding upper- and lower- 
bounds on the number of comparisons needed, as determined by the set-family structure; 
see, e.g., [6]. 

There are even more mysterious examples of multitask efficiencies in computation. Given 
a linear map T(x) : F!J — > Fg, one can ask about the complexity of circuits computing T, 
composed of F 2 -addition gates. Clearly every individual output coordinate can be computed 
by a bounded-fanin linear circuit of size n and depth [log n] , but to compute all n outputs 
simultaneously may require greater resources. However, it has been open for over 30 years 
to provide an explicit family of maps which provably cannot be computed by linear-size, 
logarithmic-depth linear circuits, even though such maps are known to exist in abundance 

H3|. 

1.1 Our Results 

Inspired by these examples, in this paper we propose and begin a systematic study of com- 
putational models from the point of view of the multitask efficiencies they exhibit. Formally 
we approach this in the following way. We fix a computational model M capable of produc- 
ing output over any finite alphabet, and a notion of cost for that model (worst-case number 
of comparisons, decision tree depth, etc.). Given a collection F = {fi(x), f 2 (x), . . . fi(x)} 
of functions on a common input x G {0, l} n , define the multitask cost function Cp{X) : 
{0, 1}' — > R by letting Cp(X) equal the minimum cost of any algorithm in M that, on input 
x G {0, l} n , outputs in some specified order the values fi(x), for every i such that Xi = 1. 
(We use capitalized variable names for vectors that index subsets of a function family F, to 
distinguish them from the lower-case vectors x which will denote inputs to F.) 

The question we are interested in is this: What kind of functions Cp(X) can arise in this 
way, when we range over all choices of Fl 

There are some obvious constraints on Cp. For many reasonable definitions of cost, Cp 
will be nonnegative and integer- valued (at least for worst-case notions of cost, which 
we will always be considering). As long as the functions in F are non-constant (and we will 
assume this throughout), Cp(X) will be if and only if X = 0. 

We expect Cp to be monotone (but not necessarily strictly monotone), since any al- 
gorithm computing a subset S C F of functions can be trivially modified to compute any 
S' C S. Finally, Cp should be subadditive; that is, we should always have Cf(X V Y) < 
Cf(X) + Cf(Y). This is because an algorithm can always solve two subcollections of func- 
tions separately and then combine the results in its output. 

Are there any other constraints? We now illustrate by example that, for at least some 
models of computation, there are functions C(X) obeying the constraints above, which do 
not correspond to Cf(X) for any choice of collection F. We consider the deterministic 
decision tree model, with depth as the complexity measure. 

For X G {0, l} 3 , let \\X\\ be the Hamming weight of X, and define 
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(0 if||X||=0, 
C*(X)=ll if \\X\ \ G {1, 2}, 
[2 if||X||=3. 

One can verify that C*(X) satisfies nonnegativity, monotonicity, and subadditivity. Now 
suppose for contradiction's sake that some family F = {fi(x), f2(x), fs( x )} satisfies Cp(X) = 
C*(X) for all X. This means that any two functions in F can be computed with one query 
to x, while it requires 2 queries to compute all three. 

Since C*(l, 1, 0) = 1, fx and f 2 must depend only on a single shared input bit Xi. Similarly 
C*(l, 0, 1) = 1 implies that fx, f% each depend on a single shared input bit Xj, so i = j. But 
then a single query to Xi determines all three functions, so that Cp(l, 1, 1) = 1 7^ C*(l, 1, 1). 
This contradicts our assumption. 

The example of C* suggests that other significant constraints might exist on multitask 
cost functions for decision-tree complexity. However, we will show that there is a strong 
sense in which this is false. In Section 12.11 we formally define economic cost functions as 
functions obeying nonnegativity (strict except at 0), monotonicity, and subadditivity; the 
rest of the paper is then devoted to proving the following result: 

Theorem 1. Given any collection 

F = {f l (x)J 2 (x),...f l (x)} 

of nonconstant Boolean functions, Cp(X) (defined relative to the adaptive query model) is 
an economic cost function. 

Furthermore, given any economic cost function C(X) : {0, 1}' — > 7L, and an e > 0, there 
exist integers n, T > 0, and a collection F = {fx(x),...fi(x)} of (total) Boolean functions 
on a common n-bit input x, such that, for all X, 

(1 - e)T ■ C(X) < C F {X) < (1 + e)T ■ C{X). 

That is, there exist multitask cost functions Cp(X) to approximate any economic cost 
function, if that economic cost function is allowed to be 'scaled up' by a multiplicative 
factor and we allow a multiplicative error of e > 0. Theorem [1] would remain true if we 
allowed economic cost functions to take non-integral values, since (up to a scaling factor) 
such functions can be arbitrarily well-approximated by integral economic cost functions. 

We summarize Theorem [T] by saying that the adaptive query model is universal for 
economic cost functions. For any model M of computation with an associated notion of 
cost, we say that M is universal for economic cost functions if the analogue of Theorem [1] is 
true with multitask cost functions from M replacing those of the adaptive query model. 

As a consequence of Theorem [TJ we obtain a universality result for any deterministic, 
adaptive model into which we can 'embed the query model'. For example of what we mean, 
let us consider the comparison model of computation over lists of integers in which a basic 
step is a comparison of two list elements (as used in the MAX/MIN result mentioned earlier). 
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Let F = {fi(x), . . . fi{x)} be any collection of Boolean functions with domain {0, l} n . 
Based on F, we define a collection G = {gi(a), . . -gi(a)} of Boolean- valued functions Qj{a) 
taking as common input a list of 2n integers a = (ai, . . . a^n)- First, let 6, = bi(a) be an 
indicator variable for the event [a2i-i < a,2i\- Then define 

9j(a) = fj{h,---b n ). 

The values 6, are each computable by a single comparison, and each pair bi, by are functions 
of disjoint variable-sets, so we see that the cost of computing any subcollection of G on a 
common input is exactly the cost (in the Boolean adaptive-query model) of computing the 
corresponding subcollection of F. 

Since the query model thus 'embeds' into the comparison model (and since cost functions 
in the comparison model can be easily seen to be economic cost functions), in light of Theorem 
[I] we conclude: 

Corollary 2. The comparison model is universal for economic cost functions. □ 

Proving such a result in the communication model seems difficult, and would require a 
better understanding of the Direct Sum phenomenon. We next state a 'Query-Model Em- 
bedding Conjecture' that would suffice to prove that the communication model is universal 
for economic cost functions, along the lines of Corollary [2j 

Let n, k > be integers. Given f(x, y) : {0, l} 2n — > {0, 1}, and a function g(z) : {0, l} k — > 
N, define a function (g o /) : {0, l} 2nk ->■ N by 

(9 ° f)(xi,Vi, x 2 , 2/2, •• • x k , y k ) = g(f(x u y 1} ),... f(x k , y k )). 

In the communication problem for go f we understand Alice to receive all x-inputs and Bob 
all y-inputs. Let cc(h) denote the (adaptive, deterministic) communication complexity of 
computing the (N-valued) function h, by a protocol in which Alice speaks first, and both 
parties learn the function value. As usual let D(g) denote the decision tree complexity of 
computing g. 

Conjecture 3. For every k 6 N and 5 e (0,1), there exists n > and a function f : 
{0, l} 2n -> {0, 1} (with cc(/) > 0; such that for all g : {0, l} fc N, we have 

cc(gof)>(l-5) cc{f)D(g). 
We can show a nearly matching upper bound 

cc(gof)<cc(f)D(g) 

for all choices of g, f, by the following protocol idea: The players consider {fej := fi(x{, yi)}i< k 
as bits to be 'queried', and simulate an optimal decision tree on these bits; whenever they 
want to determine some bj, they execute the optimal communication protocol for / on 
(xj,yj). This makes them both learn f{xj,yj), so they both know which bit 6j is to be 
'queried' next. 
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Note that the conjecture asserts a strengthened form of the Direct Sum property, for some 
particular family of functions /: by setting g to be a function that outputs an encoding of its 
input, we see that computing f(x,y) on k independent input pairs requires nearly k times 
as much communication as for one pair. 

Unable to prove the conjecture, we can at least note the following: the Conjecture really 
is sensitive to our choice of 'inner' function /. For example, let f(x,y) = x V y, and 
let g be the OR function on k bits. Then the communication complexity of computing 
(g°f) = Vi=i( x i v Vi) = (ViU x i) v (Vi=i Vi) is even though each f(x h yi) has nonzero 

communication complexity and the OR& function has decision tree complexity k. 

We suspect that a random function f(x,y), on an input size sufficiently large compared 
to k and 4, should be a suitable inner function. 

Our conjecture also appears somewhat related to the Enumeration and Elimination Con- 
jectures of pQ (so far unresolved). These are another type of variant of the Direct Sum 
Conjecture of [8]. We are not, however, aware of any formal implication between these 
conjectures and the ours. 

1.2 Outline and Methods 

To prove Theorem [TJ a key tool is the notion of hitting sets of weighted set systems. Given 
a set family A = {Ai, . . . A{\ over a universe U and a weight function w : U — > M, the 
weight of a subset B C U is defined as the sum of -B's members' weights. B is called 
a hitting set for a subfamily S C A if B intersects each G S. The hitting-set cost 
function Cx{X) : {0, 1}' — > N gives the minimum weight of any B that is a hitting set for 
S x = {A l :X l = l}. 

We use these notions to derive a useful representation lemma (Lemma [8]): for any eco- 
nomic cost function C(X) on / bits, there exists a family A = {Ax, . . . A{\ over a weighted 
universe (U,w), such that C^{X) = C(X). The (simple) proof of Lemma [8] is given in 
Section O 

As a concrete example to illustrate the expressive power of these hitting-set cost functions, 
we present a simple weighted set system whose hitting-set cost function is exactly the example 
function C*(X) presented earlier in the Introduction. (This will not be the set system that 
would be produced by our general method.) Let A be the family of all 2-element sets over 
the universe U = {wi, w 2 , %} (so, |A| = 3), and let w(ui) = 1, for each G U. Note that 
any one or two of the sets from A has a hitting set of size 1, but to hit all of A requires two 
elements. Since each element has unit weight, C&(X) is exactly C*(X). 

Returning now to the discussion of our main strategy, it will suffice to solve the fol- 
lowing problem: given a weighted set system A = {Ai, A 2 , . . . Ai}, produce a collection 
F = {f\, . . . fi} of Boolean functions over some domain {0, l} n such that Cp(X) is approxi- 
mately a multiple of Cx(X). 

Here is a high-level sketch of our collection F. For each u G U, we create a block y u of 
input variables called the 'bin' for u; x is the disjoint union of these blocks. y u represents, in 
a carefully defined way, the contents of a conceptual 'bin' which contains at most one 'key' 
k from a large set K called the 'keyspace'. 
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The bin representations and a value T > are chosen in such a way that the following 
(informal) conditions hold: 

(i) The contents of any bin y u can be determined in at most w(u)T queries; 

(ii) For any fixed k £ K and any bin y u , it can be determined with 'very few' queries 
whether k is in the bin (so that this step is 'essentially for free' in comparison to the 
queries described in (i)); 

(iii) If the number of queries an algorithm makes to the bin y u is even 'noticeably' less than 
w(u)T, the amount of information it gains about the bin contents is 'tiny', that is, 
the data seen is consistent with almost any k £ K occupying the bin. (At least, this 
outcome is unavoidable when an appropriately chosen adversary strategy determines 
the answers to queries as they are made.) 

We will formalize bins obeying the above properties in the notion of 'mystery bin func- 
tions' in Section [3721 

Returning to the sketch construction of our function collection, for i £ {1, 2, . . . I}, define 
fi(x) = 1 iff there exists some k £ K that is contained in each of the 'mystery bins' y u 
corresponding to elements u £ A+. 

To informally analyze this collection, fix any nonzero X £ {0, 1}', indexing a subcollection 
S X CA. 

For an upper bound on Cp(X), pick a minimal- weight hitting set B for Sx, so w(B) = 
Ca{X). In the first phase, for each u £ B, let our algorithm determine the bin contents of 
y u . By property (i) this phase uses at most w(B)T queries. 

Next comes the second phase. For every Ai £ Sx, there's a u £ Ai fl B, whose bin 
contents we've determined; if the bin y u was empty we can conclude fi(x) = 0. If the bin 
contained the element k £ K (remember that at most one key lies in each bin), query the 
bins of all other elements u' £ Ai to see if k is in all of them. If so, fi(x) = 1, otherwise 
fi(x) = 0. 

Thus our algorithm succeeds in computing {fi(x) : Xj = 1}. By property (ii) above, the 
query complexity of this second phase is 'negligible', giving Cp(X) < (1 + e)T ■ Ca(X) as 
needed. 

For the lower bound, we pit any algorithm using fewer than (1 — e)T • Ca{X) queries 
against an adversary strategy that runs the adversary strategies for each mystery bin in 
parallel. Since Ca{X) is the minimal cost of any hitting set for Sx, at the end of this run 
of the algorithm there must exist some Ai £ Sx such that for each u £ Ai, y u receives 
noticeably less than w(u)T queries. Using property (iii) of mystery bins, we then argue that 
the algorithm fails to determine the value fi(x). This will prove Cf{X) > (1 — e)T • Ca{X). 

The main technical challenge in implementing the above idea is to design the right repre- 
sentation of the bin contents of the blocks y u to guarantee the 'mystery bin' properties. To 
build mystery bin functions, we will exploit a small polynomial separation between decision 
tree depth and unambiguous certificate complexity, due to Savicky [llj. We describe his 
result, and reformulate it for our purposes, in Section [2731 
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How does Savicky's result facilitate our construction of 'mystery bins' ? Roughly speak- 
ing, the gap between deterministic and circuit complexity in his theorem yields the query- 
complexity gap between properties (i) and (ii) of mystery bins, while the key contribution 
of unambiguity is in allowing us to construct mystery bin functions in which the bin always 
contains at most one key In the algorithm described above to compute {fi(x) : Xi = 1}, 
this allows the query complexity of the second phase to remain negligible, yielding the upper 
bound we need on Cp{X). 

In the course of building mystery bin functions, another useful device called a 'weak 
exposure-resilient function' is also introduced and used. This object, an encoding method 
that looks uninformative when restricted to a small number of coordinates, is indeed a weak 
special case of the 'exposure-resilient functions' studied in [3]; however, the parameters we 
need are easily obtainable and so we provide a self-contained (probabilistic) construction and 
analysis. 

2 Definitions and Preliminary Results 

2.1 Vectors and Economic Cost Functions 

Given two bitvectors X = {X u . . . X t ), Y = (Y 1 , . . . Y{), we write X < Y if X, < Y h for all 
i = 1, 2, ... 7i. We define the vector Z = X V Y by the rule Z t = X { V Y t . 

Note that, in this paper, we use capital- letter variable names (X, Y, Z) to refer to vectors 
indexing 'bundles of goods', and we use lower-case variable names to refer to other vectors, 
such as the inputs and outputs to functions whose decision-tree complexity we will analyze. 

We say that a function C(X) : {0, 1}' — > Z is an economic cost function if it satisfies the 
following conditions: 

(1) C(X) > 0, and C(X) = <£> X = 0; 

(2) For all X, Y, X < Y implies C(X) < C(Y); 

(3) For all X, Y, C(X V Y) < C(X) + C(Y). 

We call such functions 'economic cost functions' due to the following informal interpretation: 
consider the input X G {0, 1} ( to C represent a certain subset of / distinct 'goods' that a 
company is capable of producing. If C(X) represents the cost to the company of producing 
one each of the goods indexed by the 1-entries of X, then intuitively, we expect C to obey 
condition (1) because there's 'no free lunch'. Condition (2) supposes that, to produce one 
bundle of goods, one can always produce a larger bundle of goods and 'throw away' the 
unwanted ones (and we assume free garbage disposal). Condition (3) supposes that, to 
produce two (possibly overlapping) bundles X, Y of goods, we can always separately produce 
the two bundles. Equality may not always hold in condition (3), even for disjoint bundles of 
goods, due to possible 'multitask efficiencies' arising in production. 

We note in passing that the definition of economic cost functions is a special case of the 
more general notion of 'outer measures' on lattices; see [2], Chapter 9. 
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2.2 Decision Trees and Multitask Cost Functions 



We will consider decision trees taking Boolean input vectors but with outputs over a possibly 
non-Boolean alphabet. A (deterministic, adaptive) decision tree T over the variables x = 
finite rooted binary tree whose internal nodes u are each labeled with some 
variable index i(u) and have designated 'left' and 'right' child nodes, and whose leaf ('output') 
nodes I are each labeled with an element v(l) G B, where B is some finite alphabet. 

A decision tree T defines a function / : {0, 1}" — >■ B in the following way: given an input 
x G {0, l} n , we begin at the root node. Whenever we are at an internal node u, we look at 
the input variable x^ u y If Xi( u ) = 0, we move to the left child of u; if xu u ) = 1, we move to 
the right child of u. Eventually we arrive at an output node /, and we define f(x) = v(l). 

We will be often consider decision trees T whose output is a bitvector: B = {0, 1}' for 
some I > 0. If the zth bit of T's output is governed by the function fi(x), we say T computes 
the collection {fi(x), f 2 (x), . . . f s (x)}. 

By the depth of T, denoted D(T), we mean the length of the longest path from the 
root in T, stepping exclusively from parent to child. Given a collection of functions S = 
{fi(x), f 2 (x), . . . fi(x)}, we define the (deterministic, adaptive) query complexity of S as 
D(S) = min {d : there exists a decision tree T of depth d computing the collection S}. US 
is a single function, S = {/}, we also write D(f) = D(S). 

We next define, for any finite collection F of functions, a function Cp which summarizes 
the multitask efficiencies existing among the members of F (relative to the decision-tree 
depth model of cost). Given a collection F of functions, F = {fi(x), f 2 (x), . . . fi(x)} on a 
common input, we define the multitask cost function Cp(X) : {0, 1}' — > Z associated with F 
by C F (X) = D(S X ), where f { e S x X t = 1. We define C F (0) = 0. 

Thus Cp{X) gives the 'cost' of certain 'bundles of goods', where cost is interpreted as 
decision tree depth, and the different 'bundles of goods' in question are the various subcol- 
lections of functions from F. As promised by part of Theorem [H we will show (Lemma [9]) 
that for any F, Cp(X) is always an economic cost function as defined in Section [27X1 

2.3 Search Problems and TUSPs 

Although in this paper we are primarily interested in the query complexity of (collections of) 
decision problems, our proof techniques also involve search problems (in the query model), 
defined next. Our definitions and terminology will be slightly idiosyncratic, but for the most 
part could be altered slightly to match up with definitions from [9]. 

Say that a string w G {0,1, *} n agrees with x G {0, l} n if for all i G [n], u>, G {0,1} 
implies Wi = X{. A search problem on domain {0, l} n is specified by a subset W C {0, 1, *}" 
called the 'witnesses'. We say that a decision tree T solves the search problem W if (i) for 
every input x that agrees with at least one w G W, T(x) outputs some w' G W agreeing 
with x (if there are more than one such w', we don't care which one), and (ii) if x agrees 
with no w G W, T(x) outputs 'no match'. 

Given a search problem W, let s(W) denote the maximum number of 0/1 entries in any 
w G W. Write D(W) to denote the minimum depth of any decision tree solving W . 
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W is called a total search problem if all x G {0, l} n agree with at least one w G W. W 
is called a unique search problem if all x agree with at most one w G W. In this paper we 
will deal with search problems W that are both total and unique; we call such a W a TUSP 
for brevity. A TUSP W defines a (total, single-valued) function from {0, 1}™ — > W mapping 
x to the unique witness w agreeing with x; we denote this function by W(x). 

For TUSPs W, as for other search problems, it is easy to see that s(W) < D(W): for 
any decision tree T solving W, the variables read by T on an input x must include all the 
0/1 entries in w = W(x). In fact, up to an at-most quadratic factor, this inequality is tight: 

Theorem 4. J3J/, ^jj, ITM/ For all unique search problems, D{W) < s(W) 2 . 

Proof. The proof is essentially identical to that of a related result, which states that decision- 
tree depth complexity is most the square of the 'certificate complexity' for Boolean functions 

0,0.1121. 

Let s = s(W). We define a query algorithm as follows: on input x, proceed in phases. At 
the beginning of phase t, let Wt C W be the set of 'live' witnesses, i.e. those that agree with 
the bits of x seen so far. Say that i G [n] is an 'active' coordinate for w G Wt if Wi G {0, 1} 
and Xi has not been queried. In each phase t, the algorithm picks an arbitrary w G Wt and 
queries x on each of the active coordinates i for w. 

Since W is a unique search problem, every distinct w, w' G W t disagree on at least one 
coordinate i active for both w and w'. Thus, in each phase t and for every w G V t , the 
number of active coordinates for w decreases by at least one. After at most s phases, then, 
no live w has any active coordinates; hence it either disagrees with x on one of the bits 
already seen, or agrees with x on each % with Wi G {0, 1}. It follows that the decision tree 
for our algorithm solves W, while making at most s + (s — 1) + . . . + 1 < s 2 queries. □ 

In 2002 Petr Savicky [11] proved a theorem implying that, in general, D(W) is not 
bounded by any constant multiple of s{W) for TUSPs. He uses different terminology and 
states a slightly different result than we need, so we will have to 'unpack' his result a little. 

A DNF formula ip is an OR of clauses, each of which consists of the AND of one or 
more literals or negated literals. Say that ijj is an unambiguous DNF (uDNF) if any input x 
satisfies at most one of its clauses. Savicky showed 

Theorem 5. J77] / There exists a family of functions 

{d : {0, 1} 4 ' {0, l}} ieN such that 

(i) Gi and Gi each have uDNF representations in which each clause has size at most 
Si = 3*/ 

(ii) D(Gi) >^ = n(s]), where 7 = log 3 (4) > 1. 

Theorem [5] is very close, but not identical, to the combination of Theorems 3.1 and 
3.6 from [TT]. That paper was concerned with the complexity measure p(f) defined as the 
minimal number of clauses in any uDNF representation of /, whereas we are concerned with 
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minimizing the maximum size of any clause as in Theorem [5j also, Savicky lower-bounds the 
number of leaves of any decision tree for / rather than its depth. However, the particular 
function family [11] gives is seen by inspection (and noted by the author) to satisfy condition 
(i), while condition (ii) follows from Savicky's lower bound on number of leaves in any decision 
tree computing G{, after noting that a decision tree with k leaves has depth at least [log (A;)] . 
this yields Theorem |5j 

We remark that it to prove our main theorem, we don't really need the full strength 
of Theorem Specifically, it would be enough that just one of G^ or d had uDNF rep- 
resentations with short clauses relative to the query complexity (or even short-clause DNF 
representations with a bounded number of satisfied clauses per input). However, using the 
full statement of Theorem [5] makes our proof slightly simpler. 

We can derive from Theorem [5] the following form of Savicky's result, which will be more 
convenient for us: 

Theorem 6. There exists a family of TUSPs 

{Wn C {0, 1, *} m( - N ^}N=i °n m(N) = O (poly (AT)) input bits, and a constant a > 0, such 
that D(W N ) > s{W N ) 1+a , while s{W N ) > N. 

Proof. For any i > 0, given uDNF representations F\, F2 of Gi and Gi respectively satisfying 
condition (i) of Theorem |5l we define a search problem Vc For every clause c in one of the 
Fj's, define a witness w c G Vi that has 0/1 entries exactly on the variables contained in c, 
with these variables set in the unique way satisfying c (remember c is a conjunction). From 
the facts that Fi,F 2 are each uDNFs and that every input x satisfies exactly one of them, 
we conclude that Vi is a TUSP. 

By condition (i) of Theorem [51 s(T^) < 3*. On the other hand, since any decision tree for 
Vi immediately yields a decision tree of the same depth for Gi, we have 

om > i±l > (i>_, 

which for large enough i is greater than s(Vi) 1+a for an appropriate constant a > 0. Also, 
by Theorems H] and 

Now we simply set W N = V r i g(iv)l+i- We verify that m(N) = A^ N ^ +1 = 0(poly(JV)). □ 

In order to make effective use of the decision-tree depth lower bound contained in The- 
orem El we will need the following folklore result, showing the optimality of the 'adversary 
method' in decision tree complexity: 

Claim 7. Let B be a finite set. Suppose f(x) : {0, l} n — > B satisfies D{h) > t > 0; then 
there exists an adversary strategy for determining the bits of x as they are queried (depending 
only on the sequence of queries made so far), such that for any query strategy making (t — 1) 
queries to x, the bits of x fixed in the process do not uniquely determine the value of f(x). 



10 



The proof of Claim [7] is a simple inductive proof by contradiction, and is omitted. Note 
that Claim [7] applies in particular when f(x) = W(x) is the (total, single-valued) function 
associated with a TUSP W. 

2.4 Set Systems and Hitting Sets 

As a final preliminary definition, we introduce hitting sets of set systems, which will play a 
key intermediate role in the proof of Theorem [U 

Given a finite 'ground set' U, and a collection A = {Ai, A2, ■ ■ ■ A{\ of subsets of U, we 
say a set B C U hits A, or is a hitting set for A, if B R Ai 7^ for all % < I. 

Given a positive function w : U — » N called a 'weight function', define the weight of a 
set A Q U as w(A) = E^^u^u). Define the weighted hitting set cost of the collection A 
(relative to w) as p(A) = min {c : there exists a hitting set i? C.U for A with w(5) < c}. 

Given a collection A = {Al, . . . Ai}, and given X G {0, 1}', define Sx = {Ai : Xj = 1}. 
Define the weighted hitting set cost function Ca(X) : {0, 1}' — > N by Ca(X) = p(Sx)- 

We now prove that the class of weighted hitting set cost functions is exactly the class of 
economic cost functions. 

Lemma 8. For any set system A and weight function w, Ca is an economic cost function. 
Moreover, given any economic cost function C(X) : {0, 1}' — > N, there exists a finite set U 
and a collection A = {Ai, . . . A{\ of subsets of U, such that for all X e {0, 1}', Ca{X) = 
C(X). 

Proof. First we show that Ca is always an economic cost function. That condition (1) of the 
definition of economic cost functions is satisfied is immediate. For condition (2), note that 
if X < Y, Sx ^ Sy, so any hitting set for Sy is also one for Sx- Thus Ca(X) = p(Sx) < 
p(Sy) = Ca(Y), as needed. 

To see that condition (3) is satisfied, note that if Bx, By are hitting sets for Sx, Sy, then 
Bx U By is a hitting set for Sx U Sy — Sxvy, and w(Bx U By) < w(Bx) + w(By). 

For the second part, let C{X) : {0, 1}' ->• N be an economic cost function. We define a 
set system and weight function as follows. Let U be a set of size 2 l , indexed by /-bit vectors 
as U = {b x : X e {0, 1}'}. 

Let A = {A u . . . A^, where A { = {b x : X { = 1}. Finally, define w(b x ) = C(X). 

We claim that, for all X = (X u . . . X t ), C A (X) = C(X). First we argue that C A (X) < 
C(X). Consider the singleton set B = {b x }- For every i such that Xj = 1, bx £ A*. Thus, 
B is a hitting set for Sx = {Ai : X^ = 1}. By definition, then, C A (X) < w(B) = w{px) = 
C{X). 

Now examine any hitting set B' for {Ai : Xi = 1}, say B' = {bz b] '■ Zy] G I C {0, 1}'}. 
For each % such that Xi — 1, Ai is hit by B' , so there exists some Zu\ G B' such that 
bz b] G Ai. Then by definition of A4, Z\j\{i) = 1- Thus X < \/ z anc ^ 

= ^ «,(6 % ) = ^ C7(Z W ) > C( V Z m ) 
z b] &i z b] ei z b] ei 
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(the last inequality holds by iterated application of property (3) of economic cost functions) 
> C(X) (since X < Vz-ez-^bl' anc ^ using property (2) of economic cost functions). Thus 
C A (X) = C(X), as claimed. □ 



3 Proof of Theorem H 
3.1 First Steps 

The first half of Theorem [1] is easy, and recorded in Lemma [9j 

Lemma 9. If F = {fi(x), /^(a?), . . . fi(x)} is a collection of nonconstant functions, Cp{X) 
is an economic cost function. 

Proof. Clearly F satisfies condition (1) in the definition of economic cost functions, since 
Cp(0) = and all decision trees computing a nonconstant function or functions has depth 
at least 1. 

Cp(X) satisfies condition (2) since, given an optimal decision tree T for computing a 
collection S = Sx of functions from S, and given a subset 5" = Sx' Q S, we can modify T 
by removing the coordinates of its output vectors corresponding to the functions in S \ S', 
yielding a decision tree T' of the same depth computing the collection S'. So D(Sx') < 
D(3 X ) and C F (X') < C F {X). 

To show that Cp(X) satisfies condition (3), let T X ,T Y be optimal decision trees of depths 
di, d 2 respectively, for computing the collections X, Y respectively. We define a decision tree 
T' as follows: we replace each output node u of Tx with a copy Ty, u of Ty, and on an output 
node v of the copy Ty, u we place the label (z(u), z(v)), where z(u) is the label of u in Tx 
and z(v) is the label of v in Ty. Then T' computes the collection Sx U Sy (possibly with 
redundant coordinates that we can remove, and up to a reordering of the outputs). The 
depth of the new tree is d\ + d%. This yields condition (3). □ 

Now we turn to the second, harder half of Theorem [TJ Following Lemma M showing the 
'universality' of hitting set cost functions, our approach to proving Theorem [1] is to build a 
collection of functions mimicking the structure of a given set system A, where each we 
create will correspond to some A{ G A. We will prove: 

Lemma 10. Given any hitting set cost function C&(X) : {0, 1}' — > N and e > 0, there exist 
integers n, T, and a collection F = {f\(x), . . . fi(x)} of functions on n bits, such that, for 
allXe{0, l} 1 , 

C A {X) ■ T(l - e) < C F (X) < C A (X) ■ T(l + e). 
In light of Lemmas [8] and [91 this will prove Theorem [lj 
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3.2 Bins and Mystery Bins 

Central to our construction of the function family of Lemma [HI] is a technical device called 
a 'bin'. A bin function is a function B(y) mapping a Boolean input y (of some fixed length) 
to subsets of size or 1 of a set K = {k%, . . . ku} called the 'keyspace'. We call the input y 
a 'bin', and say that k is 'in the bin y' if B(y) = {k}. 

Our input x to the function collection of Lemma [TU] is going consist of disjoint bins, 
one bin corresponding to each u £ U from our set system A. The bins will have different 
parameters; loosely speaking, we want the difficulty of determining the bin contents B u (y u ) 
of the bin y u corresponding to u £ U to be proportional to w(u). This property by itself 
would be relatively easy to guarantee, but we need our bins to have some other special 
properties as well, formalized next in the definition of 'mystery bins'. 

Given £ [0, 1] and an integer q > 1, say that B has security (3 for q queries, and write 
sec(B, q) > 0, if there exists an adversary strategy for answering queries to the vector y 
such that, for any query strategy making q queries to y, there exists a set H C K of size 
/3|iT|, such that for any key k £ H, the bits of y fixed in the process are consistent with the 
condition B(y) = {k}. (We do not require that the bits seen be consistent with the condition 
B(y) = 0, although the adversaries we will define in our construction do achieve this.) 

Note that in this definition, we require an adversary strategy for deciding the input bits 
as they are queried, with answers depending only on the questions and answers so far, not 
on the strategy /program making the queries. 

Fix T > 0, 5 £ (0, 1). A bin function B(y) is called a (T, 5)-mystery bin function (MBF) 
(with keyspace K), if 

(i) There is a T-query algorithm to compute B(y); 

(ii) For any k £ K, it can be decided in ST queries whether k £ B(y); 

(iii) sec(B, (1 - S)T) > (1-5). 

(Note the correspondence, when 5 is close to 0, between these conditions and their informal 
versions in the proof sketch from the Introduction.) 

Constructing mystery bin functions seems to crucially rely on a result like Theorem Eland 
its associated TUSP. Note that mystery bin functions behave quite similarly to the TUSPs 
from Theorem [6j given a particular potential witness w £ W, it is easily checked if the 
input x agrees with w; but computing W(x) may be much harder. The main additional 
ingredient in mystery bin functions is the property (iii) above, which imposes on algorithms 
a 'sharp transition' between near-total ignorance and certainty as they attempt to determine 
a bin's contents. This sharp transition is what will allow us to tightly analyze the function 
collections we will build to prove Lemma [10J 

Our construction of mystery bin functions is given by the following Lemma: 

Lemma 11. For all 5 > 0, we can find T, M > such that, for every integer c > 1, there 
exists a (cT, S) -mystery bin function with keyspace K = [M]. 
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3.3 Application of Mystery Bins 

Before proving Lemma fTTl we show how it is used to prove Lemma fTUI and, hence, Theorem 

m 

Say we are given a collection A = {A\, A 2 , . . . A{\ of subsets of a universe U, and a weight 
function w :[/—>■ N. We wish to produce a collection F — (fi, ■ ■ ■ fi) of functions such that 
the cost of computing a subset of the functions of F is approximately a fixed scalar multiple 
of the minimum cost under w of a hitting set for the corresponding sets in A. 

Let Wraax be the largest value of w(u) over U. For each u G U, we define a block of 
input y u corresponding to u and a bin function B u taking y u as input. B u is chosen as a 
(w(u)T, — 1 7| 7 ^)-MBF with keyspace K = [Ml, for some T, M > independent of u, as 
guaranteed by Lemma [TTJ Let the input x to F be defined as the disjoint union of all the 

Vu- 

For % < I, define fi(x) by 

fi(x) = 1 <^> 3k G [M] such that B u (y u ) = {k},Vu G A*. 

We claim that F satisfies the conclusions of Lemma [TU1 If X = the statement is trivial, 
so assume I/O. First we show the upper bound on Cp(X). Given the corresponding 
nonempty subset Sx Q A, let B C U be a hitting set for Sx of minimal cost: 

«;( J B)=p(5 x ) = C A (X). 
Define an algorithm Px to compute {fi{x) : Xj = 1} as follows: 

Phase 1: For each u £ B, compute bin contents B u (y u ). 

Phase 2: For every i such that Xi = 1, pick some u G B n Aj (such a w must exist, 
since 5 is a hitting set for Sx). If in Phase 1 it was found that B u (y u ) = 0, clearly 
fi(x) = 0. Otherwise, suppose B u (y u ) = {/c} for some fc G M; in this case, query 
each mystery bin B u / (y u >) such that u' G Aj, to ask whether k G B u /(y u /). By the 
definitions, fi(x) — 1 iff k is indeed the contents of all such bins, so the queries of Px 
determine fi(x) and the output nodes of Px can be labeled to compute fi(x), for every 
i with Xi = 1. 

Now we bound the number of queries made by Px- In Phase 1, each individual bin con- 
tents H u (y u ) can be computed in w(u)T queries, by property (i) of MBFs and the definition 
of B u (y u ). Then altogether, at most w(B)T = Ca(X)T queries are made in this Phase. 

In Phase 2, each question to a bin y u > asking if some k is in B u >(y u i) can be answered in 
at most 

(w(u')T) < ' ' 



w max l\U\ y y ' ' ~ l\U\ 

queries, using property (ii) of MBFs. Since at most l\U\ such questions are asked (ranging 
over (i,u')), in total at most eT such queries are made during Phase 2. Summing over the 
two Phases shows that C F (X) < D(P X ) < (1 + e)T • C A (X), as needed. 
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Now we show that Cp(X) > (1 — e)T ■ C&(X), again assuming X ^ 0. We give an 
adversary strategy to determine the bits of x as they are queried, namely: For each u G U , 
fix bits of y u as they're queried, by following the adversary strategy for B u (y u ) given by 
property (iii) in the definition of MBFs, and answer queries to y u arbitrarily if this bin 
receives more queries than the adversary strategy for ~B u (y u ) is guaranteed to handle. 

Let P be any algorithm making fewer than (1 — e)T • Ca{X) queries to the input x; we 
will show that the queries made by P against the adversary just defined fail to determine 
some value fi{x), for some i such that = 1. 

For u G U, let q u be the number of queries made by P to y u against this adversary 
strategy. Let B P = {u : q u > (1 — ^)w(u)T} . We claim B P is not a hitting set for Sx- To 



see this, note that 



w(b p ) = «>(«) < E 

udBp u£Bp 




< (1 _ f)T ((! - e ) T • c *i x )) < C A (X), 

so, by definition of C\(X) = p(Sx), Bp is not a hitting set for Sx- 

Thus there exists an i such that = 1 and such that for every u G Ai, q u < (l — ^)w(u)T. 
For each such u, by the guarantee of the adversary strategy used for bin y u , there exist at 
least 

(1 6 —)M>(l--±-)M 

W ma xl\U\ \Ai\ 

distinct keys k G [M], such that it is consistent with the bits of y u seen by P that T5 u (y u ) = 
{/••}• 

By a union bound, there exists some fixed k G K such that it is consistent with the bits 
seen that B u (y u ) = {k} for all u G Ai, which would cause fi(x) = 1. On the other hand, 
it is also clearly consistent that not all such bin contents ~B u {y u ) are equal, and hence that 
fi{x) = 0. Thus P fails to correctly compute fi(x), for at least one input x. Since Xi = 1, we 
have shown that Cf{X) > (1 — e)T • Ca(X). This finishes the proof of Lemma fTUl assuming 
Lemma [TTJ □ 



3.4 Construction of Mystery Bins 

Now we prove Lemma [TTJ First, suppose we can prove Lemma [TT] for c = 1; we'll show the 
conclusion then follows for every c G N, with the same values of T and M = \K\. 

Let B(?/) be a (T, 5)-MBF. Say the input y has length m; define a new bin function B c (?/) 
on input {0, l} cm with the same keyspace K by breaking the input y' into m blocks of size c, 
defining Zi to be the sum mod 2 of the ith block (i < m), and setting B c (y') := B(z 1; . . . z m ). 

The adversary strategy 5" for B c (y / ) is simply lifted from the strategy S for B(y), by 
answering queries in any given block i of y' as 0s until the last, 'critical' query to that ith 
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block is made, then answering this query as the strategy S would fix y^ conditioned on the 
'critical' responses made so far. Clearly any algorithm making q queries can induce at most 
\_-\ critical responses from the adversary, and so property (iii) in the definition of MBFs is 
easily seen to be inherited by B c . 

Similarly, any algorithm for determining the bin contents B(y), or for querying whether 
fc G B(i/) for some k G K, can be adapted to B c by simply querying entire blocks at a time. 
This increases the number of queries by a factor c, giving properties (i) and (ii). Thus B c (t/) 
is a (cT, <5)-MBF with keyspace [M], as needed. 

Now we prove Lemma [TT1 for the case c = 1. 

Let iV > be a (large) integer to be determined later, and let W = Wn be the TUSP 
guaranteed by Theorem[H]for parameter N, with input size m(N) = 0(poly(iV)). For brevity 
write D N = D(W), s N = s(W), and recall D N > s]^ a , s N > N. We let K := [D 2 N ] be the 
keyspace. 

We next describe the structure of the 'bin' input y. y is broken into three disjoint parts, 
written as 



• x will be an input to the TUSP W = Wn (so \x\ = m(N)); 

• WtK, called the 'witness-to-key table', will be an encoding of a function GyvtK : W —> K 
(with a specific encoding method to be described shortly); 

• KtW, called the 'key-to-witness table', will be an encoding of a function GKtw : K — >■ W 
(with a different encoding method, also described shortly). 

In our definitions, every setting to the input tables WtK, KtW will define functions 
GWk; G*Ktw as above, and such functions will generally not have unique encodings. 

Assuming for now that the two encoding schemes have been fixed, we define the bin 
function B(y) as follows: k E~B(y) if the witness w = W(x) satisfies 



Note that at most one key can be in the bin by this definition (or the bin may be empty). 

Now we describe the encodings. KtW simply uses any efficient encoding with a table entry 
Kt\N\k corresponding to each element k of the domain K. Since each w G W C {0, 1, *} m ^ Ar ) 
has at most sn 0/1 entries, \W\ cannot be too large, namely 



since m(N) = 0(poly(iV)). Thus each table entry KtW|^ in KtW can be represented 
using 0(sn log(iV)) bits. We do so, assigning 'leftover' codewords arbitrarily to elements of 
W, so that every table defines a function (and also every function is representable). 



y = (x, WtK, KtW), where: 



G WtK (w) = k, Gxtw(fc) = w- 
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For the encoding WtK, we want table entries to be 'obfuscated', so that it takes many 
queries to learn anything about an individual value of GWk- We make the following defini- 
tion, which resembles more- demanding definitions in [3]: 

Fix integers m,d,t > 0. Say that a mapping J : {0, l}' m — >■ [d] is an (m,d,t)-weak 
Exposure- Resilient Function (wERF) if for every c G [d] and every subset S C [m] of size at 
most t, there is a b G {0, l} m with J(b) = c, and such that the entries of b indexed by S are 
all-zero. 

Claim 12. For sufficiently large N > 0, there exists a ( L s ]v" J , D 2 N , \_\s l ^ a ^ 2 \ )-wERF J. 

Proof. Let J be a uniformly chosen random function from the domain {0, l} m (with m = 
[s]^ a ^ 2 \) to the range [d] = [D 2 N ]. We show that with nonzero probability J satisfies the 
definition of an (m, d, t)-wERF with t : = [\s l + a/2 \. 

Fix any subset S C [m] of size t, and a c G [d]. We analyze the probability ps, c that 
there is no b G J -1 ^) such that b is all-zero when restricted to the coordinates in S. This 
is simply (1 — i) 2 ™ . Now by our settings, for sufficiently large N we have m — t > d2 m / 3 . 
Thus for such N, 

I 1 \ nm/3 

Taking a union bound over all choices of S, c, the probability that J fails to be an (m, d, t)- 
wERF is, for large enough N, less than 2 m de~ 2m/ = o(l). So, with nonzero probability we 
succeed. □ 

Recall that in our setting K = [D 2 N ]. We let each table entry WtK^ of WtK (with 
position indexed by a witness w G W) contain L s iv^ Q//2 J bits, and define 

GWkM = J(WtK| w ), 

where J is as given by Claim [121 

This completes our description of the bin function B(y). We now show that for a large 
enough choice of N it is a ((1 + |)Djv,5) mystery bin function. 

First we verify property (i) in the definition of MBFs. In order for a query algorithm 
to determine the bin contents B(y), it suffices to do the following: Inspect x to determine 
w = W(x); look up (jwtK(w), finding some key k; finally, check to see if (jKtw(fc) = w. If so, 
B(y) = {k}, otherwise the bin is empty. 

The first step can be implemented in D N queries to x. For the second step, table entries 
of WtK are of size |_s]y +_CE ' /2 J , which is o{Dn) since Dn > s]^~ a . The third step, querying a 
table entry of KtW, takes 0(sn log(iV)) queries, which is also o{D^). Thus the total number 
of queries is Dn(1 + o(l)), less than (1 + f )-Djv for large enough N. This shows property (i). 

For property (ii) of MBFs, let k G K be any key; to determine if {k} = B(y), our 
algorithm queries KtW| fc to find w = (?Ktw(&) and, subsequently, queries \NtK\ w to determine 
if k = G\/\/ t K(w). If not, then {k} ^ B(y), and the algorithm reports this. If k = GwtK^), 
then the algorithm makes at most sjy = o(D N ) queries to x to see if x agrees with w. Note 
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that each step takes o(D N ) queries, smaller than 5(1 + ^)D N for large N. This gives property 
(ii). 

Finally, we show property (iii). This is the property for which we will use the fact that \K\ 
is large and entries of WtK are'exposure-resilient'. Our adversary strategy against algorithms 
making at most (1 — 5)(1 + |)-Djv < (1 — §)A/v queries to y is as follows: 

• Answer queries to x according to a strategy, guaranteed to exist by Claim [JJ that pre- 
vents any query strategy making fewer than Dn queries to x from uniquely determining 
the value W(x). Answer all queries to WtK, KtW with zeros. 

Our proof of correctness is by contradiction. Suppose some deterministic algorithm P 
makes at most (1 — f )-Djv queries to y against this adversary, and afterwards outputs a list 
L of fewer than (1 — S)\K\ keys, such that the bin contents B(y) is forced by the bits seen 
to either be empty or contain a key from L. 

Define a new algorithm P' as follows: in Phase 1 P' first simulates P on y, making all 
the queries P does. After P terminates, define V C W as the set of all witnesses w for which 
P has made more than Ll 5 ^"^] queries to the table entry \NtK\ w in WtK. In Phase 2, for 
each w G V in turn, P' makes any additional queries to x necessary to determine whether x 
agrees with w. 

Say this latter set of queries in Phase 2 are 'on behalf of w\ Note that for every w G V, 
at most sjy queries are made on behalf of w in Phase 2, while more than L^a^ 2 ] queries 
are made to the table entry WtK^ in Phase 1. It follows that only an o(l) fraction of the 
queries of P' are made in Phase 2, hence for large enough N, P' makes fewer than queries 
to y. 

But we claim that P' succeeds in determining W(x), contrary to the guarantee of our 
adversary strategy from Claim [71 First, after the simulated operation of P by P', say that 
a witness w is 'live' if the bits of x seen are consistent with the possibility W(x) = w. Note 
that if there is a live witness w whose table entry in WtK has been queried at most L^tv^ 2 ] 
times, then the value GwtK^) is completely undetermined (any value is consistent with the 
bits seen), since the adversary answered those queries with zeros and the function J used in 
defining G WtK (w) is a ( [s]^ a/2 \ , D%, [\s^ al2 \ )-wERF. 

Thus, for any key k whose table entry in KtW was not queried by P, it is consistent with 
the bits of y seen that B(y) = {k}. Since \K\ = D 2 N = uj(Dn), if N is sufficiently large then 
P cannot query a bit from even a 5 fraction of KtWs table entries. Hence, for P to output 
the list of candidates L C K with \L\ < (1 — §)\K\, it must be that for every w G W still live 
after the operation of P, the WtK entry for w must have been queried more than LIstv^ 2 ] 
times, and thus w G V. 

Since no two distinct w G W are compatible, it follows that exactly one w remains live 
after Phase 2 of the operation of P', so P' determines the value W(x). Again, this is in 
contradiction to the guarantee of our adversary strategy from Claim UJ so the assumption 
about P was false. We have proved that B(y) satisfies property (iii) in the definition of 
MBFs, and altogether we have shown that B(y) is a ((1 + |)-Dat, 5)-MBF, proving Lemma 
EU for c= 1 (with T = (1 + %)D N ,M = D%). " ' □ 
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